The Effect of Rating Method on Reliability of Judgments of Strain Across Populations.

PURPOSE: Variability in auditory-perceptual ratings of voice limits their utility, with the poorest reliability often noted for vocal strain. The purpose of this study was to determine whether an experimental method, called visual sort and rate (VSR), promoted stronger rater reliability than visual analog scale (VAS), for ratings of strain in two clinical populations: adductor laryngeal dystonia (ADLD) and vocal hyperfunction (VH). METHOD: Connected speech samples from speakers with ADLD and VH as well as age- and sex-matched controls were selected from a database. Fifteen inexperienced listeners rated strain for two speaker sets (25 ADLD speakers and five controls; 25 VH speakers and five controls) across four rating blocks: VAS-ADLD, VSR-ADLD, VAS-VH, and VSR-VH. For the VAS task, listeners rated each speaker for strain using a vertically oriented 100-mm VAS. For the VSR task, stimuli were distributed into sets of samples with a range of severities in each set. Listeners sorted and ranked samples for strain within each set, and final ratings were captured on a vertically oriented 100-mm VAS. Intrarater reliability (Pearson's r) and interrater variability (mean of the squared differences between a listener's ratings and group mean ratings) were compared across rating methods and populations using two repeated-measures analyses of variance. RESULTS: Intrarater reliability of strain was significantly stronger when listeners used VSR compared to VAS; listeners also showed significantly better intrarater reliability in ADLD than VH. Listeners demonstrated significantly less interrater variability (better reliability) when using VSR compared to VAS. No significant effect of population or interactions was found between listeners for measures of interrater variability. CONCLUSIONS: VSR increases intrarater reliability for ratings of vocal strain in speakers with VH and ADLD. VSR decreases variability of auditory-perceptual judgments of strain between inexperienced listeners in these clinical populations. Future research should determine whether benefits of VSR extend to voice clinicians and/or clinical settings.

Test-Retest Reliability of Behavioral Assays of Feedforward and Feedback Auditory-Motor Control of Voice and Articulation.

PURPOSE: Behavioral assays of feedforward and feedback auditory-motor control of voice and articulation frequently are used to make inferences about underlying neural mechanisms and to study speech development and disorders. However, no studies have examined the test-retest reliability of such measures, which is critical for rigorous study of auditory-motor control. Thus, the purpose of the present study was to assess the reliability of assays of feedforward and feedback control in voice versus articulation domains. METHOD: Twenty-eight participants (14 cisgender women, 12 cisgender men, one transgender man, one transmasculine/nonbinary) who denied any history of speech, hearing, or neurological impairment were measured for responses to predictable versus unexpected auditory feedback perturbations of vocal (fundamental frequency, fo) and articulatory (first formant, F1) acoustic parameters twice, with 3-6 weeks between sessions. Reliability was measured with intraclass correlations. RESULTS: Opposite patterns of reliability were observed for fo and F1; fo reflexive responses showed good reliability and fo adaptive responses showed poor reliability, whereas F1 reflexive responses showed poor reliability and F1 adaptive responses showed moderate reliability. However, a criterion-referenced categorical measurement of fo adaptive responses as typical versus atypical showed substantial test-retest agreement. CONCLUSIONS: Individual responses to some behavioral assays of auditory-motor control of speech should be interpreted with caution, which has implications for several fields of research. Additional research is needed to establish reliable criterion-referenced measures of F1 adaptive responses as well as fo and F1 reflexive responses. Furthermore, the opposite patterns of test-retest reliability observed for voice versus articulation add to growing evidence for differences in underlying neural control mechanisms.

Clinical Cutoff Scores for Acoustic Indices of Vocal Hyperfunction That Combine Relative Fundamental Frequency and Cepstral Peak Prominence.

PURPOSE: This study examined the discriminative ability of acoustic indices of vocal hyperfunction combining smoothed cepstral peak prominence (CPPS) and relative fundamental frequency (RFF). METHOD: Demographic, CPPS, and RFF parameters were entered into logistic regression models trained on two 1:1 case-control groups: individuals with and without nonphonotraumatic vocal hyperfunction (NPVH; n = 360) and phonotraumatic vocal hyperfunction (PVH; n = 240). Equations from the final models were used to predict group membership in two independent test sets (n = 100 each). RESULTS: Both CPPS and RFF parameters significantly improved model fits for NPVH and PVH after accounting for demographics. CPPS explained unique variance beyond RFF in both models. RFF explained unique variance beyond CPPS in the PVH model. Final models included CPPS and RFF offset parameters for both NPVH and PVH; RFF onset parameters were significant only in the PVH model. Area under the receiver operating characteristic curve analysis for the independent test sets revealed acceptable classification for NPVH (72%) and good classification for PVH (86%). CONCLUSIONS: A combination of CPPS and RFF parameters showed better discriminative ability than either measure alone for PVH. Clinical cutoff scores for acoustic indices of vocal hyperfunction are proposed for assessment and screening purposes.

The Effect of Visual Sort and Rate Versus Visual Analog Scales on the Reliability of Judgments of Dysphonia.

Purpose The reliability of auditory-perceptual judgments between listeners is a long-standing problem in the assessment of voice disorders. The purpose of this study was to determine whether a relatively novel experimental scaling method, called visual sort and rate (VSR), yielded stronger reliability than the more frequently used method of visual analog scales (VAS) for ratings of overall severity (OS) and breathiness (BR) in speakers with voicedisorders. Method Fifty speech samples were selected from a database of speakers with voice disorders. Twenty-two inexperienced listeners provided ratings of OS or BR in four rating blocks: VSR-OS, VSR-BR, VAS-OS, and VSR-BR. For the VAS task, listeners rated each speaker for BR or OS using a vertically oriented 100-mm VAS. For the VSR task, stimuli were distributed into sets of samples with a range of speaker severities in each set. Listeners sorted and ranked samples for OS or BR within each set, and final ratings were captured on a vertically oriented 100-mm VAS. Interrater variability, defined as the mean of the squared differences between a listener's ratings and group mean ratings, and intrarater reliability (Pearson r) were compared across rating tasks for OS and BR using paired t tests. Results Results showed that listeners had significantly less interrater variability (better reliability) when using VSR methods compared to VAS for judgments of both OS and BR. Intrarater reliability was high across rating tasks and dimensions; however, ratings of BR were significantly more consistent within individual listeners when using VAS than when using VSR. Conclusions VSR is an experimental method that decreases variability of auditory-perceptual judgments between inexperienced listeners when rating speakers with a range of dysphonic severities and disorders. Future research should determine whether a clinically viable tool may be developed based on VSR principles and whether such benefits extend to experienced listeners.

A Randomized Controlled Trial of Two Semi-Occluded Vocal Tract Voice Therapy Protocols.

PURPOSE: Although there is a long history of use of semi-occluded vocal tract gestures in voice therapy, including phonation through thin tubes or straws, the efficacy of phonation through tubes has not been established. This study compares results from a therapy program on the basis of phonation through a flow-resistant tube (FRT) with Vocal Function Exercises (VFE), an established set of exercises that utilize oral semi-occlusions. METHOD: Twenty subjects (16 women, 4 men) with dysphonia and/or vocal fatigue were randomly assigned to 1 of 4 treatment conditions: (a) immediate FRT therapy, (b) immediate VFE therapy, (c) delayed FRT therapy, or (d) delayed VFE therapy. Subjects receiving delayed therapy served as a no-treatment control group. RESULTS: Voice Handicap Index (Jacobson et al., 1997) scores showed significant improvement for both treatment groups relative to the no-treatment group. Comparison of the effect sizes suggests FRT therapy is noninferior to VFE in terms of reduction in Voice Handicap Index scores. Significant reductions in Roughness on the Consensus Auditory-Perceptual Evaluation of Voice (Kempster, Gerratt, Verdolini Abbott, Barkmeier-Kraemer, & Hillman, 2009) were found for the FRT subjects, with no other significant voice quality findings. CONCLUSIONS: VFE and FRT therapy may improve voice quality of life in some individuals with dysphonia. FRT therapy was noninferior to VFE in improving voice quality of life in this study.